APPROXIMATION OF CHAOTIC OPERATORS 
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Abstract. As well-known, the concept "hypercyclic" in operator theory is 
the same as the concept "transitive" in dynamical system. Now the class 
of hypercyclic operators is well studied. Following the idea of research in 
hypercyclic operators, we consider classes of operators with some kinds of 
chaotic properties in this article. 

First of all, the closures of the sets of all Li-Yorke chaotic operators or 
distributionally chaotic operators are discussed. We give a spectral description 
of them and prove that the two closures coincide with each other. Moreover, 
both the set of all Li-Yorke chaotic operators and the set of all distributionally 
chaotic operators have nonempty interiors which coincide with each other as 
well. The article also includes the containing relation between the closure of the 
set of all hypercyclic operators and the closure of the set of all distributionally 
chaotic operators. Finally, we get connectedness of the sets considered above. 



1. Introduction 

We are interested in the dynamical systems induced by continuous linear op- 
erators on Banach spaces. From Rolewicz's article [21], hypercyclicity is widely 
studied. In fact, it coincides with a dynamical property "transitivity". Now there 
has been got so many improvements at this aspect, Grosse-Erdmann's and Shapiro's 
articles [8, 23] are good surveys. 

In his celebrated work [9, 10, 11], D. A. Herrero studied the chaotic properties 
(hypercyclic and Devaney's chaotic) of linear operators. It is important since it 
shows that we can study the chaotic properties of operators in a really operator 
theory way. As well-known, it is hard to check whether a topological system be 
chaotic or not for a general object. But following Herrero's idea, we can use the 
technique of approximation to study the properties of chaotic operators on Hilbert 
space under compact or small perturbation. An interesting result, obtained by D. 
A. Herrero and Z. Y. Wang [9] or K. Chan and J. Shapiro [3], shows that the identity 
operator / can be perturbed by a small compact operator to be hypercyclic. This 
stronger result implies that a small perturbation of a simple operator can be an 
operator with complex dynamic properties. 

These papers suggest us to consider the following question: 
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Question : Which kinds of operators can be approximated by chaotic operators? 

From the point of approximation, we should consider closure of the set of all op- 
erators satisfying some chaotic property. In this paper, Li-Yorke chaotic operators 
and distributionally chaotic operators will be studied by classical approximation 
tools developed in [12]. 

In order to explain the main results, we must introduce some definitions and 
properties of chaos and Hilbert space operators. 

In 1975, Li and Yorke [16] observed complicated dynamical behavior for the class 
of interval maps with period 3. This phenomena is currently known under the name 
of Li-Yorke chaos. Recall that a discrete dynamical system is simply a continuous 
mapping / : X —* X where X is a complete separable metric space. For x € X, 
the orbit of x under / is Orb(f, x) — {x, f(x), f 2 (x), . . .} where /" = / o / o • • • o / 
is the n th iterate of / obtained by composing / with n times. 

Definition 1.1. {x,y} C X is said to be a Li-Yorke chaotic pair, if 
Iimsupd(/ n (a;),/ n (y)) > 0, liminf d(f n (x), f n (y)) = 0. 

n — >oo n *°° 

Furthermore, / is called Li-Yorke chaotic, if there exists an uncountable subset 
TCX such that each pair of two distinct points in T is a Li-Yorke chaotic pair. 

In 1994, Schweizer and Smrtal [22] gave the definition of distributional chaos 
(where it was called strong chaos), which requires more complicated statistical 
dependence between orbits than the existence of points which are proximal but not 
asymptotic. 

For any pair {x, y} C X and any n <G N, define distributional function F xy : R — ► 
[0,1]: 

F: y (r) = ^#{0 < i < n - 1 : d{f{x),f{y)) < r}, 
where is the cardinality of the set A. Furthermore, define 

F xy (r) = liminf F^ y (r), 
F* y (r) = limsupF^(r) 

n— ►oo 

Both F xy and F* are nondecreasing functions and may be viewed as cumulative 
probability distributional functions satisfying F xv (t) = F* y (r) = for r < 0. 

Definition 1.2. {x,y} C X is said to be a distributionally chaotic pair, if 

F* y (r) = 1, V t > and F xy (e) = 0, 3 e > 0. 

Furthermore, / is called distributionally chaotic, if there exists an uncountable 
subset ACI such that each pair of two distinct points in A is a distributionally 
chaotic pair. Moreover, A is called a distributionally e-scrambled set. 
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From the definitions, we know distributional chaos implies Li-Yorke chaos. But 
the converse implication is not true in general. In practice, even in the simple case 
of Li-Yorke chaos, it might be quite difficult to prove chaotic behavior from the very 
definition. Such attempts have been made in the context of linear operators (see 
[6, 7]). Further results of [6] were extended in [20] to distributional chaos for the an- 
nihilation operator of a quantum harmonic oscillator. Additionally, Distributional 
chaos for shift operators were discussed by F. Martmez-Gimenez, et. al. in [19]. 
More about Li-Yorke chaos and distributional chaos, one can see [1, 17, 18, 24, 25]. 
In a recent article [13], B. Hou et. al. introduced a new dynamical property for 
linear operators called norm-unimodality which implies distributional chaos, and 
obtained a sufficient condition for Cowen-Douglas operator being distributional 
chaotic and Devaney's chaotic. We introduce the definition of norm-unimodality 
here. 

Definition 1.3. Let X be a Banach space and let T £ B(X). T is called norm- 
unimodal, if we have a constant 7 > 1 such that for any m £ N, there exists x m £ X 
satisfying 

lim ||T fe x m || = 0, and ||T l a; m || > 7*||a; m ||, i = 1, 2, . . . , m. 

Furthermore, such 7 is said to be a norm-unimodal constant for the norm-unimodal 
operator T. 

Next, we introduce the notations and properties of Hilbert space operators. Let 
H be complex separable Hilbert space and denote by B(H) the set of bounded 
linear operators mapping H into H. For T £ B(H), denote the kernel of T and the 
range of T by KerT and RanT respectively. Denote by <r(T), a e (T), <Ji re (T) and 
<7 W (T) the spectrum, the essential spectrum, the wolf spectrum and the weyl spec- 
trum of T respectively. For A £ p s - F {T) := <C\a lre (T), ind(A — T) = dimKer(A - 
T)-dimKer(A-T)*,min ind(A-T) = min{dimKer(A - T), dimKer(A - T)*}. De- 
note p n s _ F {T) = {\£ Ps-f(T); ind(A - T) = n}, where -00 < n < 00, pj_ F {T) = 
{A £ p s - F (T); ind(A - T) > 0} and p~_ F (T) = {X £ p s - F (T); ind(A — T) < 0}. 
According to [12] corollary 1.14 we know that the funtion A — ► min ind(A — T) is 
constant on every component of p s -p(T) except for an at most denumerable subset 
p s s _p(T) without limit points in p s _p(T). Furthermore, if p, £ p s s _ F (T) and A is 
a point of p s _ F (T) in the same component as p but A is not in p s s _ F (T), then 
min ind(/i — T) > min ind(A — T). p s s _ F (T) is the set of singular points of the 
semi-Fredholm domain p s -f{T) of T; p r s _ F (T) = p s _ F (T)\p s s _ F (T) is the set of 
regular points. Denote by ao(T) the set of isolated points of a(T)\a e (T). Denote 
by E and E°, the closure and the interior of set E respectively. In addition, denote 
by LY(H), DC(H), UN(H) the set of all Li-Yorke chaotic operators, the set of all 
distributionally chaotic operators and the set of all norm-unimodal operators on H 
respectively. 
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Now we are in a position to state the main results of this article. In section 
2, the closures and interiors of the sets of all distributionally chaotic operators or 
Li-Yorke chaotic operators are considered. Though distributionally chaotic opera- 
tors require more complicated statistical dependence between orbits than Li-Yorke 
chaotic operators, we have 

I. DC(H) = LY{H) = {T e B(H); dB n er (re (T) ^ 0} U {T e B(H); <9D C 
Ps-f{T) and dimKcr(A - T) > 0, VA <G <9D} (Theorem 2.8). 

II. DC(H)° = LY(H)° = {Te B(H), 3 A e dB s.t. ind(A-T) > 0} (Theorem 
2.14). 

From the above two results, one can see distributionally chaotic operators and 
Li-Yorke chaotic operators are very similar. The closure of DC(H)° (i.e. the closure 
of LY(H)°) is also considered. 

III. DC(H)° = LY(H) a = {Te B(H); <9D g p ( °2 F (T) U p^l F {T)}. Moreover, 
DC(H)\DC(H) a = LY(H)\LY(H)° = {Te B(H); <9D C p { °} F {T) U p { s Z F {T) and 
dimKcr(A-T) > 0, VA G dB}U{T e B(H); dBOo lre {T) ^ and p^} F (T) n dU> — 
0}. (Theorem 2.16). 

In section 3, we get the relation between hypercyclic operators and distribu- 
tionally chaotic operators. In detail, the set of all hypercyclic operators belongs 
to the closure of DC(H)°. The relation between norm-unimodal operators and 
distributionally chaotic operators is also obtained. 

IV. UN{H) = DC{H) = LY(H), DC(H) a = LY(H)° C UN(H), and 
UN(H)\DC{H)° = DC{H)\DC{H)° = LY(H)\LY(H)°. (Theorem 3.3). 

It follows from this result that, the norm-unimodal operators are very large in 
the class of distributionally chaotic operators. Moreover, it is useful for people to 
prove that an operator is distributionally chaotic as the criterion of hypercyclic 
operators given by Kitai [15] and refined by Grosse-Erdmann and Shapiro, et.al. 
[8]. 

In section 4, we consider the connectedness of the sets considered above. 

V. DC{H)°, DC(H)°, DC{H) and DC(H)\DC(H)° ( i.e. LY(H)°, LY(H)°, 
LY(H) and LY(H)\LY(H)° ) are all arcwise connected. (Theorem 4.1). 

2. Closures and interiors of the sets of all distribitionally chaotic 
operators or ll-yorke chaotic operators 

Firstly, we need some lemmas which will be used in the proof theorem 2.8. The 
definition given by Cowen and Douglas [4] is well known as follows. 

Definition 2.1. For a connected open subset of C and n a positive integer, let 
B n (il) denotes the operators T in B(H) which satisfy: 

(1) Q C a(T); 

(2) ran(T — uS) = H for lj in fl; 

(3) y ujen kev(T-u) = H; and 
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(4) dimker(T — u>) = n for u> in f2. 

One often calls the operator T in B n {Q) Cowen-Douglas operator. Denote by D 
and <90 the unit open disk and its boundary. Then we have the following theorem. 

Theorem 2.2. [13] Let T G B n (Q). If fl n <9D ^ <j>, then T is norm-unimodal. 
Consequently, T is distributionally chaotic. 

Remark 2.3. In fact, this result can be extended to n = oo. 

Lemma 2.4. Let T G B{H). Then the following statements are equivalent. 

(1) T is not Li-Yorke chaotic. 

(2) liminf ||T n (a;)|| = implies lim ||T n (a;)|| = 0. 

n — *oo n — >oo 

The proof is easy and left to the reader. 
Lemma 2.5. Let T G B{H), a{T) n dD = 0. T/zen liminf ||T n (x)|| = implies 

n — >oo 

lim ||T"(x)|| = 0. Moreover, T is neither Li-Yorke chaotic nor distributionally 

n— ►oo 

chaotic. 

Proof. Since a(T) n <9D = 0, then according to Riesz's decomposition theorem 
where ct(Ti) = cr(T) n D and <t(T 2 ) = cr(T) - <t(7i). Furthermore, 



# 2 ' 



where T 2 ~ T 2 and then a(T 2 ) = a(T 2 ) = a(T) - a(Ti). 
By spectral mapping theorem and spectral radius formula, 

n(^)" 1 = r(^ _1 ) = lim ||7b" n ||s, (where n(-) = inf{|A|; A G c(-)}). 

n — ►oo 

Notice ri(T 2 ) > 5 > 1, then there is e > such that ri(T 2 ) _1 + e < 1. Hence we 
have M G N such that for any n > M, 

-^>(_J )». 

\\T 2 n \\ > 1 (T 2 )-i+e ; 

Furthermore, for any y G Hi' 1 , 

\\T2 n (y)\\ > — U-lll/ll > (^rr^ ) n Hfll ^ when n ^ M - 

\\T 2 || n(T 2 )-i + e 

Let liminf || T™ (a;) 1 1 = and x = x\ x 2 , xi G H\, x 2 G -Hi -1 , then one can 

n — >oo 

easily obtain x 2 = and then T n (x) = T™(a;i). On the other hand r(T\) < 1, so 
there exist < p < 1 and N G N such that for any n> N, ||27*(a;i)|| < /o n ||xi||. 
Therefore, lim \\T n (x)\\ = lim ||Ti"Oi)|| = 0. □ 

n^oo n^oo 

Following from Kitai's result in [15], no finite dimensional Hilbert space supports 
a hypercyclic operator. In fact, we have 
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Lemma 2.6. Let < n< oo be an integer and T £ B(C n ). Tften liminf ||T m (a;)|| = 

m— *oo 

implies lira ||T m (a;)|| = 0. Moreover, T is neither Li-Yorke chaotic nor distri- 
butionally chaotic. 

From [14], one can see 

Lemma 2.7. For any e > 0, there is a compact operator K e £ B(H) such that 
\\K e \\ < e and I + K e is distributionally chaotic. 

Now we will give a description of the closures for the sets of distributionally 
chaotic operators or Li-Yorke chaotic operators. 

Theorem 2.8. Let E x = {T £ B(H)\ dD n ai re {T) + 0} and E 2 = {T £ 

B(H); dB C p s - F {T) and dimKcr(A - T) > 0, VA £ «9D}. Then DC(H) = 
LY{H) =EiU4 



Proof. Clearly, DC{H) C LY(H). So it suffices to show E x U E 2 C DC{H) and 
LY{H) <ZE 1 \J E 2 . 

First step, E 1 U E 2 C DC{H). We will show for any T £ E X \J E 2 and e > 0, 
there exists an operator C such that ||C|| < e and T + C £ DC(H). In fact, one 
can obtain a compact operator K such that \\K\ \ < e and T + K £ DC(H). 

If T £ Ei, then choose a A £ dB> n oi re {T). By AFV theorem there exists a 
compact operator Ki such that ||i^i|| < e/2 and 

T+^ 1= [ Ao/ *' 

where dimi? = 00 • 

Following lemma 2.7 there exists a compact operator K 2 such that H-K2II < e /2 
and Aq/ + K 2 is distributionally chaotic. Let 



H 
H ^ 



K 2 







Ho 
Ho ± 



Then T+ifi+i ; i:2 G DC(H), where i^i+i^ is a compact operator and || J K"i+/f 2 || < 
e. 

If T £ E 2 , define 

H r = \/ Ker(A-T), 

Aep;_ F (T)nA 

where A is the component of semi-Fredholm domain p s _p(T) which contains dD. 
Then dimH r = 00 and 



Since 



(1) p r s _ F (T) n A C a(T r ), 

(2) dimKer(^i - T r ) = dimKer( / u — T)=n, V/i £ p r s _ F {T) n A, 
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where n € N + U {oo}, 



(3) \/ Ker(n-T r )= \f Ker(p - T) = H r , 

Mep^_ F (T)nA p.ep r s _ F (T)nA 

(4) Ran(/i - T r ) = H r , p e p r s _ F {T) n A, 

we know T r e B n (p r s _ F (T) n A). Notice p r s _ F (T) n A n <9D ^ 0, by theorem 2.2 
T r is norm-unimodal and hence distributionally chaotic. So is T. The first step is 
complete. 

Second step, LY(H) C^U^. Notice 

{E 1 U E 2 } c = {T e B(H); dB C p s _ F (T) and 3A G <9D s.t. dimKcr(A — T) = 0}, 

by Fredholm theory {E 1 U £ 2 } c is open. Since {LY(H)} C = {LY(H) C }°, it suffices 
to prove {E 1 U E 2 } c C LY(H) C . 
Let T e {-Ei U £ 2 } c , define 

^ = V/ Ker(A - T)*, 

where is the component of semi-Fredholm domain p s ^ F (T) which contains dD. 
Then 



T * 
Ti 



Hf 
H, 



, (H t maybe {0}!). 



Claim 1: p r s _ F (T) n $ C p(T ). 

Let /i <G p r s _ F (T)n^. Since A — > min ind(A— T) is constant on the semi-Fredholm 
domain p r s _ F (T) and 3A € 3D s.t. dimKer(A -T) = 0, we have Ker(p-T) = {0}. 
Hence Ker(/i-T ) = {0}. Notice Ker(p-T)* = Ker(p-Ti)* and (p-T)*(Hi) = H u 
then Ker(/x — To)* = {0}. Therefore, p — To is invertible. 

Claim 2: a (T ) n $ = <j(T ) n $ = p s s _ F {T) n 

From claim 1, ct (T ) n $ C cr(T ) n $ C p s s _ F (T) n Let A e p s s - F {T) n 
If A — To is invertible, then A — 1} is a semi-Fredholm operator and min ind(A — 
Ti)=min ind(A - T). Since (A - Ti)*(H{) = H u dimKer(A - T t ) = and hence min 
ind(A-T)=min ind(A-7j)=0. It is contradict to A € p s s _ F (T). Therefore, A-T is 
not invertible and p s s _ F (T) (1 $ C cr(T ) n <&. Because A — T is a left semi-Fredholm 
operator, (A — T)(H^-) is closed. Therefore A — T is a semi-Fredholm operator. 
Again notice claim 1 we obtain p s s _ F (T) n <f> C a (T ) n <I>. 

Since the only limit points of p s s _ F (T) belong to d[p s _ F (T)], let cr (T ) n <i> n 
<9B = {/ii}™ 1; m < oo. By Riesz's decomposition theorem and Rosenblum-Davis- 
Rosenthal corollary [12], 



To 



Too 


Hoo _ 


Too 


* 


Hoo 


Too 


Hoo 


T i_ 


Hoi 






Hi G -ffoo 


Toi_ 


Hi Q H o 
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where a(T 00 ) = {^}™ i, m < oo, a(T 01 ) n dB = 0, T Q1 ~ T i and dimfto < oo. 
Hence 



T~S:-- 



'Too £ * ' 
T i * 
21 



ft)0 

ft © ftra- 

ft 



Moreover ft = V Ker ( A — T)* = V K er(A - T)*, then 

Ae P j_ F (T)n* Ae9Dnpj_ F (T)n* 



Ai 

* A2 

* * A3 



ei 

62 

e3' 



where {AJ^ C»n p r s _ F {T) n $ and {ej^ is an ONB of ft. 

Now we come to end the proof. Since Li-Yorke chaos is invariant under similar 
and lemma 2.4, it suffices to show liminf ||5 n (x)|| = implies lim = 0. 

Let liminf \\S n (x) \ \ = 0, then there exist {rik}'?, such that lim ||5™ fe (x)|| = 

n— >oo ' rife— >oo 

0. Notice x = x x~o x;, x G fto, xo € ft^ e fto, x ; e ft, we have 
lim ||T, rafc (x;)|| = 0. Following the matrix representation of 7], xj = 0. Hence, 

rifc^oo 



^(x) 



too 



1 01 



So by lemma 2.6 and lemma 2.5 lim ||Toq(xo)|| = and lim 

n — >oo n — >oo 

That is lim ||5 n (x)|| = 0. The second step is complete. 



£01 



(*o) 



0. 

□ 



Theorem 2.8 also includes the information of the interior for DC(H) C . Obviously, 
the operator T satisfying, a(T) DdD = 0, is in {DC(H) C }° (i.e. {DC(H)} C ). There 
exists an operator T in {DC(H) C }° , whose spectrum a(T) intersects the unit circle, 
i.e. <t(T) n <9D ^ 0. 

Example 2.9. Let A e B(H) satisfying 

{Aei = |e i+ i, i < -2, 

Aei = 2e i+ i, i > -2, 

where {eJ^L^ is an ONB of H. Then A is in {£>C(ft) c }°. 

Proof. Through easy compute, one can obtain cr(A) = {z e C; 5 < |z| < 2} and 
ind(A - A) = -1, dimKer(A - A) = for A e {z e C; | < |z| < 2}. According to 
theorem 2.8, A e {L>C( J ff)} c (i.e. {£>C(ft) c } ). 
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We consider the dynamical property of A. For any x in H, x = T,°l_ 00 Xiei and 



A 2n+1 (x) = ( 



A 2n (x) = ( 



1 1 1 

22n+l X -( 2 ™+ 2 )' 22ri-l X -(2«+ 1 )' 22n-3 X - 2n > 



-X. 



-(n+2)j 2a;_( n+ i), 2' 



x 



-ni ' " " ); 
1 



1 1 

' 22^ X ~2n-l: 22n-2 X - 2n > 2 2n ~± X -2«+l i 



where the position under A is the position corresponding to the ONB {ej}"^^. 
One can easily obtain if x ^ 0, then — + oo. Hence A is not distributionally 
chaotic. □ 

Next, we consider the interiors of the sets of all Li-Yorke chaotic operators or 
distributionally chaotic operators. Before proving the result theorem 2.14, it is 
convenient to cite in full length a result of Apostol and Morrel. Let T — dil, where 

is an analytic Cauchy domain, and let L 2 (T) be the Hilbert space of (equivalent 
classes of) complex functions on T which are square integrable with respect to 
(l/27r)-times the arc-length measure on T; M(T) will stand for the operator defined 
as multiplication by A on L 2 (T). The subspace H 2 (T) spanned by the rational 
functions with poles outside f2 is invariant under M(T). By M + (T) and M_(T) 
we shall denote the restriction of M(T) to -ff 2 (T) and its compression to L 2 (T) Q 
H 2 (r), respectively, i.e. 



M(r) = 



M+(T) 



z 

M_(r) 



H 2 (T) 

# 2 (ry 



Definition 2.10. [12] S € B(H) is a simple model, if it has the form 



s = 



A 



where 

(1) cr(5+), <t(S-), <j(A) are pairwise disjoint; 

(2) A is similar to a normal operator with finite spectrum; 

(3) S+ is (either absent or) unitarity equivalent to ©™ 1 M + (dfli) ki , 1 < ki < oo, 
where {dSli}™ =1 is a finite family of analytic Cauchy domains with pairwise diajoint 
closures; 

(4) S- is (either absent or) unitarity equivalent to ®™ =1 M-(d$j) hj , 1 < hj < oo, 
where {<9<J>,, }™ =1 is a finite family of analytic Cauchy domains with pairwise diajoint 
closures. 
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Theorem 2.11. [12] The simple models are dense in B(H). More precisely: Given 
T £ B{H) and e > there exists a simple model S such that 

(1) a(S + ) C P ;_ F (T) C a(S+) e , a(S.) C C <t(S_) £) and <r(A) C 
a(T) e . 

(2) ind(A -S) = ind(A - T), /or each A e /o7_ F (S+) U p+_ F (S-). 

(3) ||T-S||<6. 

Additionally, we give a lemma which appeared in another article [14]. But for 
convenience to read this article, we also give the details of the proof. 

Lemma 2.12. Let N £ B(H) be a normal operator. Then liminf ||iV™(x)|| = 

71— *00 

implies lim ||./V n (x)|| = 0. Moreover, N is neither Li-Yorke chaotic nor distribu- 
tionally chaotic. 

Proof. Since TV is a normal operator, then there exist a locally compact space X, 
a finite positive regular Borel measure /x and a Borel function r\ £ L°°(X,p:) such 
that N and M v are unitarily equivalent. M v is multiplication by 77 on L 2 (X,p). 
Let liminf || M™(/) 1 1 = and 

-co ' 

A 1 = {z£X;\ V (z)\ > 1}, 

A 2 = {z£X;\ V (z)\ < 1}, 

A 3 = {z£X;f(z)=0 a.e. [jj]}, 

A 4 = {z£X;f(z)^0 a.e. \p]}. 

Then there exists a sequence of positive integers {rifcl^j such that lim ||M™ fe (/)|| = 

rifc-^oo 

and 

||M^(/)H 2 = / \v nk f\ 2 d^ 

= f \v nk f\ 2 d»+ [ \v nk f\ 2 d» 

JAinA 4 JA 2 nA 4 

> / l/| 2 ^+ I \v nk f\ 2 d». 

JAinA 4 JA 2 nA 4 

Consequently fi(A\ n A4) = 0. For any n £ N, there exists a positive integer k such 
that rife < n < rik+i- Therefore, 



K?(/)l| 2 = / \v n f\ 2 dn 
JA 2 nA 4 

J A 2 nA 4 

< / \v nk f\ 2 dv 
JA 2 nA 4 

■ I 2 



= iiM»*(/)r, 

and hence lim ||M"(/)|| = 0. Notice the property which we considered is invariant 
under unitarily equivalence, we obtain the result. □ 
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Corollary 2.13. Let T E B(H) be a subnormal operator. Then liminf ||T"(x)|| = 

n— >oo 

implies lim ||T n (x)|| = 0. Moreover, T is neither Li-Yorke chaotic nor distribu- 

n^oo 

tionally chaotic. 

Theorem 2.14. Let F = {T E B{H), 3 A E dD s.t. ind(A - T) > 0}. Then 
DC(H)° = LY(H)° = F. 

Proof. Obviously, DC(H)° C LY(H)°. So we only need to show F C DC(H)° and 
LY(H)° C F. 

First step, F C DC(H)°. By Fredholm theory, F is open. Consequently, it 
suffices to prove F C DC(H). 
Let T E F. Define 

ff r = \/ Ker( M -T), 

Mep^_ F (T)nA 

where A is the component of /9^ F (T) which contains a point in 9D. Then dimi7 r = 
oo and 



T = 



* 

Similar to theorem 2.8, we have T r E B n (p r s _ F {T) fl A), moreover T is norm- 
unimodal and distributionally chaotic. The first step is complete. 



Next step, LY(H)° C F. Because {LY(H)°} C = LY(H) C , we only need to show 
for any T E F c and e > 0, there exists C E B{H) such that ||C|| < e and T + C is 
not Li-Yorke chaotic. 

Let T E F c and e > 0. According to Theorem 2.11, there exists a simple model 

S+ * * 
A * 

such that a(S-) C pf_ F (T) C a(S-) e and \\T - S\\ < e. Since p^ F (T) n 9D = 0, 
we can ensure a(S_) D 9D = according to the proof of theorem 2.11. So it suffices 
to prove S is impossible to be Li-Yorke chaotic. 

Notice a(S + ), <t(5_), <t(A) are pairwise disjoint, according to Rosenblum-Davis- 
Rosenthal corollary [12], 

A 

S_ 

Because Li-Yorke chaotic is invariant under similar and A is similar to a normal 
operator N with finite spectrum, we directly let S = S+ ® N ® S_. 

If liminf \\S n (x) \ \ — 0, since x = x + © x$ © x_ corresponding to the space 

n^oo 

decomposition, following lamma 2.5 and corollary 2.13 we have 

\S+ 



lim ||S*„™0_)|| = and lim 



N 



x+ 
x 



= 0. 



Hence lim ||5 n (x) 



0. S is not Li-Yorke chaotic. The second step is complete. 

□ 
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We give an example which is distributionally chaotic but not in DC(H) . 

Example 2.15. Let A e B(H) satisfying 

{Ae i = 2e i - 1 , i ^ 0, 

Ae = 0, 

where {ei}fl_ 00 is an ONB of H. Then A is distributionally chaotic but not in 
DC{H)°. 

Proof. Since Ho = V°^ { e i} is an invariant space of A and A\h = 2B is distri- 
butionally chaotic, where B is backward unilateral shift, then A is distributionally 
chaotic. One can easily obtain ind(A — ^4) = for |A| < 2, so A is not in DC(H)°. 

We prove it directly. For any e > 0, let K S B(H) satisfying Keo = ee_i, Kei — 
0, i ^0. Then K is compact and ||.K|| = e. Since a{A + C) = {z e C; \z\ = 2}, 
A + K is not distributionally chaotic. Hence ^4 is not in DC(H)°. □ 

Unfortunately, the closure of DC(H)° (i.e. the closure of LY(H)°) is not equal 
to the closure of DC(H) (i.e. the closure of LY(H)). It means there exists a class 
of distributionally chaotic operators (Li-Yorke chaotic operators) which are more 
complicated. We give these descriptions. 

Theorem 2.16. Let G = {T e B(H); dD £ p { °2 F (T) U p^T)}, Gi = {T e 
B(H);dD C p ( ?2 F (T)\Jp£) F (T) and dimKer(A-T) > 0, VA e dD} and G 2 = {T E 
B(H);dD n a ire (T) ^ and p ( s %(T) n dD = 0}. Then DC(H)° = LY(H)° = G 
and DC(H)\DC(H)° = LY(H)\LY(H)° = Gi U G 2 . 



Proof. First, we prove DC(H)° = LY(H)° = G . Clearly, DC(H)° = LY(H)° = 
F C Go, where F is denoted in theorem 2.14. By Fredholm theory, Go is closed. 
Hence DC(H)° = LY(H)° C G . So we only need to show for any T e G and 
e > 0, there exists G e such that | |G| | < e and T + G € DC(H)° (i.e. 

LF(if) ). 

Let T G Go and e > 0. Then 

(1) 3A e <9D s.t. ind(A — T) > 0, or 

(2) ^+^(7) n <9D = 0, but 3A e 3D s.t. ind(A - T) = 0, or 

(3) bi+k T ) U p1-f(^)] n 9D = 0, but a ire (T) n 8D ? 0. 
Case (1) is obvious. 

Case (2). First it implies <7; re (T) n <9D ^ 0. Then choose a A € cr (re (T) n 
<9[p^ F (T) n dD]. According to AFV theorem, there exists a compact operator K\ 
such that 1 1 .KiH < e/2 and 



T+K x 



X I * 
A 



Ho 



Ho 
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where a (A) = a(T), <ri re (A) = a lre (T) and ind(A - A) = ind(A - T), for A e 
Ps-f{T). Let 



B f = 



e/2 

e/2 



ei 
e2 



where {ej}^ is an ONB of TJq an d 



^ = 







Ho 1 - 



< such that md(T+K 1 + K 2 -X) 



Obviously, | j i + 1 j < e and there exists A e 
1 > 0. Hence T + K 1 + K 2 ^ DC(H)°. 

Case (3). By theorem 2.11 there exists C\ such that ||Ci|| < e/2 and 

T + d 



* 
.4 



where S+ is either absent or unitarity equivalent to a subnormal operator and 
dV)\cr(S+) contains a small arc in <9B, A is similar to a normal operator with finite 
spectrum and ai re (A) n <9B ^ 0, S- is either absent or unitarity equivalent to the 
adjoint of a subnormal operator and cr(S-) n 9D = 0; a(S+), cr(A), cr(S-) are 
pairwise disjoint. Furthermore, 

a, re (T + Ci) = <T lre (S + )Ua lre {A)Ua lre (S_) and p(T + d) = p{S + )r\p{A)r\p(S_). 
Hence 

(T ire (T + Ci) n dD ^ and p(T + C\) n 3D ^ 0. 
Then we can obtain C2 through the technology of case (2) such that 1 1 C2 1 1 < e/2 
and ind(T + C x + C 2 - A x ) > (where A x G dB>). Hence T + Ci + C* 2 e DC(H)°. 
The first equation is complete. 

Second, we prove DC(H)\DC(H)° = LY(H)\LY(H)° = G 1 U G 2 . Clearly, 

DC(H) \ DC(H)° C DC(H)\DC(H)° C LY(H)\LY(H)° C LY{H) \LY(H)°. 



Then d C DC(H)\DC(H)° C LY(H)\LY(H)° C d U G 2 . In order to obtain 
the result, we only need to show G 2 C DC(H)\DC(H)°. 

For any T e G 2 and e > 0, according to AFV theorem there exists a compact 
operator ifi such that ||.Ki|| < e/2 and 



Aq/ * 



where Ao € <9B n cr; re (T). By lamma 2.7, we know there exists a compact operator 
K e such that \\K e \\ < e/2 and Ao/ + K e is distributionally chaotic. Let 





#2 



#0 



H 



So + K 2 \ \ < e and T + K\ + K 2 is distributionally chaotic. Notice 

p { s + J f (t + K! + k 2 ) n do = P [ + \{T) n do = 0, 
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we know T + K\ + K 2 £ {DC(H)°} C . The second equation is complete. □ 

3. Some other results 

In this section, we consider the relation between hypercyclic operators and dis- 
tributionally chaotic operators, and the closure of the set of all norm-unimodal 
operators. 

Recall the definition of chaos given by Devaney [5] as follows. 

Definition 3.1. Suppose that / : X — > X is a continuous function on a complete 
separable metric space X, then / is Devaney's chaotic if: 

(a) the periodic points for / are dense in X, 

(b) / is transitive, 

(c) / has sensitive dependence on initial conditions. 

It was shown by Banks et. al. [2] that if / satisfies (a) and (6), then / must 
have sensitive dependence on initial conditions. Hence only the first two conditions 
of the definition need to be verified. 

Denote by HC(H) and DE(H) the set of all hypercyclic operators and the set of 
all Devaney's chaotic operators on H respectively. Obviously, DE(H) C HC(H). 



Proposition 3.2. DE{H) = HC(H) C DC(H) a = LY(H)°. 



Proof. According to [11] proposition 4 and [10], one can obtain DE(H) = HC'(H) = 
{T £ B(H); a w (T)UdB is connected, a (T) = and ind(A-T) > 0, A £ p s _ F (T)}. 
Following theorem 2.16, we obtain the result. □ 

Next, we can see the set of all norm-unimodal operators is large in the set of all 
distributionally chaotic operators. 

Theorem 3.3. UN(H) = DC(H) = LY(H), DC(H)° = LY(H)° C UN(H) and 
UN(H)\DC(H)° = DC(H)\DC(H)° = LY(H)\LY(H)°. 



Proof. First we prove UN(H) = DC(H) = LY(H). Obviously, UN(H) C DC(H) = 
LY(H), it suffices to prove E\ U E 2 C UN(H), where E\, E 2 are denoted in the- 
orem 2.8. We will show for any T E E\ U E 2 and e > 0, there exists C such that 
||C|| < e and T + C E UN(H). But different to theorem 2.8, we can not generally 
find a compact operator satisfying the property. 

If T £ Ei, then choose any Ao £ dDD ai re (T). According to AFV theorem, there 
exists a compact operator K\ such that H-fS'iH < e/2 and 



T+K 1 

where dimTJo = oo. Let 

C x = 



X I * 



H 



e/2 
e/2 



eo 
ei 
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where {ei}°l is an ONB of Ho- Then C\ is a Cowen-Douglas operator with | \C\ 
e/2. Let 

C\ H 




Hn 



+ < e). 



C 1 = 

Following theorem 2.2, we know 

T+K 1 + Ch E UN(H), 

Notice Ki + Ci is not compact. In fact, the operator T in Ex satisfying a(T) = 
ai re (T) and a(T) C B~ can not be perturbed into UN(H) by compact operator, 
since T + K is impossible to be norm-unimodal (one can observe [14] for details). 

If T e E2, then according to the first step in the proof of theorem 2.8, we 
know T is norm-unimodal. The first equation is complete. The second inclusion is 
immediate obtained from the first step in the proof of theorem 2.14. 

Next we prove UN(H)\DC(H)° = DC(H)\DC(H)° = LY (H)\LY (H)° . Clearly, 
UN(H)\DC(H)° C DC(H)\DC{H)° = LY (H)\LY (H)° . We only need to show 
Gi U G 2 C UN(H)\DC(H)°, where G u G 2 are denoted in theorem 2.16. Similar 
to the first step of theorem 2.8, Gi C UN(H)\DC(H)°. 

Let T € G2 and e > 0. By AFV theorem there exists a compact operator K\ 
such that \\K\\\ < e/3 and 







Ao/ * * 


H 


T + K ± 




A * 


Hi, 






A /_ 


H 2 


where A G <9Dn er (re (T), p i s h) F (A)ndB = 0. Let N e B(i? 2 ) be an uniform infinite 


multiplicity normal operator such that a(N) = {z e C; |z| < e/3} and 




"0 


e/3 


eo 






e/3 


ei 

5 



where {ejg is an ONB of H . 

Then ||JV|| = e/3; \\B e \\ = e/3, B e e Si(D e/3 ) and a(S e ) - D £/3 - 
Hence 



Notice 







"0 




X I + B £ * * 





+ 







A * 







N 




A / + 7V 



o {+) ( 

Ps-F\ 



= [p { s %(A)\{zeC; \z-\ \ <e/3}]n, 



XqI + B e * * 
A * 
X Q I + N 



)n. 



and Aq/ + -B e is norm-unimodal, we obtain the result. 



□ 
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Example 3.4. Let A € B(H) satisfying 



Ae = e_i, 
Ae- - ^ 



ei_i, 



i > 1, 



i < -1. 



where {eJ^L^ is ONB of H. Then A is norm-unimodal, but not in DC(H)°. 
Proof. For any m e N, ||A*(e m )|| > 2 i ||e m ||, 1 < i < m and lim ||A n (e m )|| = 0. 

n — >oo 

So A is norm-unimodal. Since a(A) = {z e C; 1 < |z| < 2} implies <9B C ai re (A), 
then A is not in DC(H)°. □ 

4. Connectedness 

The main purpose in this section is to discuss the connectedness for the sets 
considered in section 2. 



Theorem 4.1. DC{H)°, DC(H)°, DC{H) and DC(H)\DC(H)° (i.e. LY(H)°, 
LY(H)°, LY(H) and LY(H)\LY(H)° ) are all arcwise connected. 

Proof. We show DC(H)° is arcwise connected, others are similar. 

First step, for any T <s DC(H)°, it can be connected to f e DC(H)°, where 
<rire(T) n <9D ^ 0. If a lre (T) n dD ^ 0, then obviously. 

Let o- lre (T) n <9D = 0. Then <9D C p^ F (T). Choose A e cr/ re (T), by AFV 
theorem there exists a compact operator K such that ||X|| < e and 

A()/ * * 
T+K= A * 

where cr(T) = cr (re (r) = er (re (A) and ind(A - T) = ind(A - A) for all A e 

p s _ F (A). Choose n € <9D, let 

a(i)I * * 
5{t) = A * , 1 < t < 2, 

a(t)J 

where a(i) = (t - 1)(mo - A ) + A , 1 < t < 2. Define 



m = | 

Obviously, 

(3(0) = T, 0(2) = T := 



Hal * * 
A * 
Vol 



< t < 1, 

1 < f < 2. 

a ire (f)naD = { Alo }^0 



and /3(f) is continuous on [0,2]. 

For any 1 < t < 2, p^W*)) = p£k^)\M*)}> then pi+ F (<5(*)) n dB + 0. 
Notice p£ F (T + tA") = ^+ F (T) D <9D for all < f < 1, we know {/3(f); < f < 
2} C DC(H)°. The first step is complete. 
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Second step, we show for any T,S £ DC(H)°, a We {T)C\dO ^ 0, <7i re (S)C)dB ^ 
T and S can be connected. 

Let A e d[p^} F (T) n SO] be the point such that there exists O > s.t 
{X e ie ; < < O } C p^ F (T). Similarly, we can obtain Ai e n 



and 6»i > such that {Aie ie ; < < 0i} C (0 *+ ) J? (S'). Notice there exists 



0' such that A = e w A x , define S* = e ie '£. Then A E d[p ( s %(S) n 3D] and 
{A e ie ; < < 6 X } C p^ F (S) (let $ be the component of p s _ F (S) which con- 
tains {A e* e ; < < 0i}). 

By AFV theorem and theorem 3.48 [12], for any e > 0, there exist compact 
operators K\, K 2 such that \\K\\\ < e, H-f^H < e and 



A / Cy 

A x 



H x 



Hi 



1, S + K 2 = 



A 2 C 2 
Xol 



H 2 ^ 
H 2 



where a{A 1 ) = a(T), ai re {Ai) = a lre (T) and ind(A - Ai) = ind(A - T) for all 
A G Ps-f(T); a{A 2 ) = a(S), a lre {A 2 ) = a lre {S), ind(A - A 2 ) = ind(A - S) for all 
A e p s -p(S) and minind(A — A 2 ) = for all A e <f> - Without loss of generality, let 
H x = H 2 ^. Define 

e u S, 



7(*) 



< t < , 

S + (t-0')jr 2 , 0'<i<0' + l, 

(t-(0' + l))(T + X 1 ) + (0' + 2-i)(5 + X 2 ), 0' + l<t <0'+2, 
I T + [0' + 3 - t]K u e' + 2 < t < e' + 3. 



Obviously, 7(0) = 5, 7(0 + 3) = T and 7(f) is continuous on [0, +3]. We prove 
{7(t); < t < 0' + 3} C DC(H)°. 

a) p^e^S) = e'Vi-k 5 ')^ < i < 0' and p^p^ndB ^ implies {e^S; < 
t < 0'} C DC(H)°, 

b) pi+kS + (t-fl')^ 2 ) = pi+U^) = e'Vi-Us), 0' < t < 0' + l and p<+> F (S)n 
<9D^0 implies {S+{t-e')K 2 ; 0' < t < 0' + 1} C DC(H)°, 

c) For any given + 1 < i < +2, since 

p1-f([* - (0' + l)]Ao + [#' + 2 - i]A 2 ) = [t - (0' + 1)]A + [0' + 2 - t]p[ + _} F (A 2 ) 



and 



pi-Ul* - («' + + [«' + 2 - i]A ) = [t - (0' + l)R%(^i) + [9 + 2- i]Ao 

= A o +[<-(0' + l)](^+ ) i? (yl 1 )-Ao), 

we know there exists t > such that 

{Aoe 49 ; < < t } 

Cpi+k[* - («' + 1)1*0 + [0' + 2 - i]A 2 ) n - (0' + 1)]A, + [6+2- t]A ). 



A o + (0' + 2-i)(p(+ F (A 2 )-Ao), 



(+) 
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For each A € {A e ie ; < < t }, notice minind(/i — A 2 ) = for fi e $0, so 
minind([i - (6' + 1)]A + [0' + 2 - t]A 2 - A) = 0. Moreover, ind([i - (<?' + 1)]A + 
[6' + 2 - t]A 2 - A) > 0, then we have [i - (0' + 1)]A + [0' + 2 - t]A 2 - A is epic. 
Consequently 

ind([i - (0' + 1)](T+ Jfi) + [9 + 2 - + K 2 ) - A) 
=ind([i - (0' + 1)]A + [9 + 2 - t]^ 2 - A) + 
ind([t - {8 + l)]A x + [8+2- t]Ao - A) 
> 0. 



Hence {(i - (0' + 1))(T + Xi) + (&' + 2- t){S + K 2 ); 9' + 1 < t < 9' +2} C DC(H)°. 

d ) /0^k T +^'+ 3 -i]^i) = /°i-k T )> e '+ 2 < t < 9' +3 and p^ F (T) ndB + 
implies {T + [0' + 3 - t\K^ 9' + 2 < t < 9' + 3} C DC(H)°. 

Therefore, {-y(t); < t < 9' + 3} C DC(H)°. The second step is complete. 
Thus, DC(H)° is arcwise connected. □ 

Example 4.2. Let i? be backward unilateral shift. Then there exists an arc a(t) 
in DC(H)° which connects 5B and 5.B 2 . 

Proof. Through easy compute, a(5B) = a(5B 2 ) = 5D~ and ind(A— 5B) = 1, ind(A— 
5B 2 ) = 2 for |A| < 5. First there exist compact operators K\, K 2 such that 



55 + ^1 



-51 Ci 



#i 



#i 



1 and hB z + K 2 



A 2 C 2 
51 



H 2 
H 2 



where a(A\) = <r(5B), ai re (Ai) — ai re (5B) and ind(A — A\) = ind(A — 5B) for 
all A e p s - F (5B); <j(A 2 ) = <r(5B 2 ), <J lre {A 2 ) = ai re (5B 2 ) and ind(A - A 2 ) = 
ind(A - 5B 2 ) for all A e p s -f(5B 2 ). Without loss of generality, let Hi = i? 2 ± - 
Define 



5B + (l + t)Jfi, 
5B 2 + (2-t)if 2 , 



-1 < t < 0, 

< i < 1, 

1 < f < 2, 



where 8(t) 



, < t < 1. 



— implies 



-5{l-t)+tA 2 (l-t)d+tC 2 
(1 - t)A 1 + 5t 

Obviously, a(— 1) = 5_B, a(2) = 5B 2 and a(i) is continuous on [-1,2]. It suffices 
to show a(t) e DC(H)° for any -1 < i < 2. 

(1) p1-f( 5B + (1 + *)#i) = pi_k 5B ). -1 < * < and 
{5B + (1 + -1 < * < 0} C DC(H)°. 

(2) pitk*(*)) = bi-k-5(l - *) + ^2)] U [pi+k(l - *)^i + 5*)] = {-5(1 - t) + 
5iD} U {5(1 - i)B + 5i}, < i < 1 implies {<5(i); < t < 1} C DC(H)°. 

One can read spectrum properties from the picture as follows. The number 1 
or 2 in the picture means the index of the open disk which it lies respectively. We 
choose five moments. 
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(3) pY_ ] F {bB 2 + {2-t)K 2 ) = p\ + _ ] F {hB 2 ), 1 < t < 2 and <9D C p ( s t' F (5B 2 ) implies 
{5B 2 + (2 - t)K 2 - 1 < t < 2} C DC(H)°. 

Hence {a(i), -1 < t < 2} C DC(H)°. □ 
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